In data storage and data transmission one of the challenges is to reduce the amount of data to be stored and the amount of data to be transmitted. If one is willing to accept that some data is lost, one may cluster instances of the data and represent the instances of the data that belongs to a single cluster with a shorter description that uniquely identifies the cluster and use the shorter descriptions instead of the longer instances of the data. Instead of “forming clusters of instances of the data” one may read grouping instances of the data, binning instances of the data or quantizing instances of the data. Representing the instances of the data with clusters is most useful if instances of the data having similar properties with respect to at least one characteristic end up in the same cluster.
If, for example, the instances of the data are used to predict an instance of another type of data, one would like to put instances of the data that most probably result in the same prediction in a single cluster. Thus, the clusters must be formed in such a way that the mutual information between the instances of the another type of data and the original instances of the data is maintained as far as possible in the process of clustering the instances of the data.
The paper “Quantization with an Information-Theoretic Distortion Measure” of Jean Cardinal discloses a method using a modification of Lloyd's algorithm for finding a quantizer of data X such that the mutual information between X and related Y does not much reduce as the result of the quantizing. The document “Quantization with an Information-Theoretic Distortion Measure”, Jean Cardinal, Oct. 23, 2002, is published by the “Université Libre de Bruxelles” on the website http://www.ulb.ac.be/di/publications/RT_2002. html, and is also published on the website http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.20.3058.